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1. INTRODUCTION 

The World Health Organization (WHO) reports that about 253 million people suffer from vision 
impairment; among which 36 million are blind. Most of the people of them are low-income over the eightieth 
area unit aged fifty years [1]-[3]. Vision impairment humans perpetually rely on the chosen person in their 
daily life. It is pretty arduous for the blind man to go out without help, find out a house, subway stations, and 
so on. However, there is not a decent device to help them within the least throughout an inexpensive value. If 
the visually handicapped person gets a tool form of an addict forever with him and offers essential directions, 
they go to realize accumulated independence and freedom. With the proposed guiding glass, we can give the 
visually impaired individual necessary information. Here the guiding glass means pair of eyeglasses that 
contain the hardware parts of our system. We inserted the hardware part in the glass because it is placed on 
the eyes whose working principle is similar to the eye. 

We found many works that give buzzer vibration, obstacles detection [4], [5], multisensory strategy 
[6] output when any object comes in front of the blind man. A guiding cane also generate voice instructions 
for blind individuals [7]. A walker contains an ultrasonic sensor [8], microcontroller integrated circuit (MIC) 
[9], vibrator which helps blind people during walking [10]. We find not any device that notifies blind people 
about hole information. In this respect, our contribution is to give a smart device to the blind individual 
which is very efficient in detecting any hole objects and provides specific audio output. We also integrate 
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global positioning system (GPS) technology into our tool so that it is more helpful for a blind individual. 
There are lots of methods for detecting obstacles nowadays. Based on the sensing type, it can be laser-based, 
ultrasonic-based, infrared (IR), and image processing-based [11]. The sonar detector has a drawback; it 
cannot confirm the precise direction of going forward. Optical lasers are additionally used for impediment 
recognition for cellular robots [12]. Some strategies use a mono-camera, stereo camera, and red green 
blue-depth (RGB-D) digital camera [13], [14]. Some methods use deep learning-based object detection [15]. 
For the image process [16], other ways are exploitation lately with an associate in nursing OpenCV, artificial 
neural networks (ANN), and support vector machine (SVM). 

In this work, we used the TensorFlow [17] and faster region-based convolutional neural network 
(R-CNN) [18], [19] inception v3 model whose working principle is very similar to the MobileNet model. 
Inception v3 [20] networks limit the model size and computational cost. Another important feature of our 
model is location tracking via GPS module. Location tracking is used in an automobile for tracking positions. 
We can sketch a car anti-theft gadget primarily based on global system for mobile communications (GSM) 
and GPS modules. The device is developed based totally on the high velocity of a one-chip C8051F120 and 
became aware of the vehicle [21] stolen to the car owned by using the vibration sensor. The caring role can 
be completed through the GPS gadget connected to the anti-theft system. Then, the proprietor of the car 
identifies the robbed car and gets back to his stolen car by using GPS technology. 


2. PROPOSED MODEL ARCHITECTURE 

The proposed guiding glass architecture of our guiding glass is shown in Figure 1. Given the system 
design diagram includes a Raspberry pi camera module on the module’s catches, the video stream rounds the 
blind, and it was sent to the processing unit. It also contains a NEO6M GPS module, two ultrasonic sensors 
angle at ninety and thirty degrees severally. It additionally carries a headphone. 


GPS Module 


Figure 1. Hardware structure of the proposed smart guided glass 


The calculations of hole and obstacle distance are shown in Figure 2. Here the distance is calculated 
using an ultrasonic device sloping at 30 degrees along with the frame. The first measurement was done for a 
person whose height was 64 inches (165 cm). The sonar sensor to ground distance was Y=165cos30 
(143 cm), and the blind man-to-hole distance was X=165tan30 (6 cm). When the distance was more than 
145 cm, our proposed system sent audio instruction that a hole in front of the blind man about 95 cm to the 
ears. When the distance was 140 cm or less than 140 cm, then the smart glass had notified the blind man of a 
barrier in front of you. It observed 2 cm to any deeper before the visually impaired individual. An equivalent 
ultrasonic detector angle at 30 degrees had observed any style of a barrier. Its detection precision was very 
high. We experimented with different people of different heights in Table 1 and tested the efficiency of the 
device. It worked correctly in most situations. The general expression of barrier and hole detection is shown 
in (1): 


Y = Xcos@ (1) 
where Y is the distance sonar to the ground, and X is the height of the blind man. Here 8 is kept constant at 30 
degrees. As we measured any type of holes and barrier distance by using 30 degrees angled sonar sensor, it is 
also important to calculate the obstacles in front of the visually impaired person. 90 degree angled sonar 


sensor calculates the distance parallel to the ground. Our system was effective for a distance of about 300 cm. 
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So, it could notify the blind individual as “An obstacle in front of you about d cm”. Then object recognition 


process recognized the object. Table 1 represents the change of distance to the sonar and sensor to ground 
associated with different height person. 


Sonar (30° angle) 


Zac 
= X=165xtan30"= 95.2cm 


Blind Person Barrier / Obstacle emer aed 


Figure 2. Sketch representation of hole and barrier detection 


Table 1. Represents different heights associated sonar to hole and barrier minimum distance 


SI # Height (cm) (X) Sonar to ground distance (cm) (Y) Sonar to hole distance (cm) Sonar to barrier distance (cm) 
1 182.88 (6°) 158 >160 <156 
2 177.69 (5°10”) 154 >156 <152 
3 165.45 (5’4’’) 143 >145 <140 
4 157.58 (5°2’) 136 >138 <134 
5 152.4 (5°) 132 >134 <130 
6 142.32 (4’8”’) 123 >125 <121 


Another important feature is object recognition and provides associated audio information to the 
blind man’s ears. In our proposed work, the TensorFlow framework was employed to acknowledge all types 
of objects [22]. TensorFlow is an open-source and American Standard Code for Information Interchange 
(ASCII) textual content file software program gadget library for dataflow and differentiable programming 
tasks. Here, the inception network was used for object recognition in TensorFlow. Inception network strategy 
is to avoid bottlenecks and directly allowing information flow through the network. It gives vital frameworks 
for neural network configuration. During these frameworks, ANN are described by procedure graphs. For 
beholding, 1* initialized the pi camera, and it took concerning one minute to initialize. Then the grabbed 
video frame compares with the TensorFlow library info. Google TensorFlow library info contains one 
million object data. It had obstacle size, shape, length, and specific feature data. When our module collected a 
picture of the obstacles, then the Raspberry pi 3 modules were compared with the TensorFlow library info 
and predicted what object before the blind person. The object recognition algorithmic program is shown in 
Figure 3. 

Our guiding glass can transfer location information to the family member when needed. A NEO6M 
GPS module can detect the latitude and longitude value and send it to the specific mail. We attached a 
switched beside the guiding glass and when the blind individual felt that he/she wanted to send position 
information to the family member, then pressed the button. A mail was sent to a specific person, and it 
carried out latitude and longitude values. Then family members could identify the blind individual position 
via Google map. It can detect area coordinates from four satellites. 

Figure 4 represents the schematic layout of four satellites and the GPS receiver system. The GPS 
measurement is associated with 4 equations with 4 unknowns x, y, z, and te [23]. Where te is the time 
correction, x, y, z is the receiver’s coordinates acquired from GP. The equations are given in (2): 


dı = elta — tr +t) = V0- 4} +O- + E-z) 
dz = C(ti2 — tr2 + te) = V(X — x2)? + (Y — y2} + (2-22)? 
d3 = C(tt3 — tr3 + te) = NE = x3)? + (y = y3)? + (Z — z3)? 


dy = C(tt4 — tra + te) = y (x — x4)? + (Y — Y4)? + (Z — 24)? (2) 


where c is known as the speed of light, i.e., 3x108 m/s. ty, t2, t3, ta times that GPS satellites 1, 2, 3, and 4, 
respectively, transmitted their signals (these times are provided to the receiver as part of the information is 
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sent). t1, t,2, tr3, tr4 times when signals are received from GPS satellites one, two, three, and four. x), yı, zı 
GPS coordinates directly from satellites. The receiver of the GPS module solves these equations concurrently 
to found out the value of x, y, z, and fc. 
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Figure 4. Schematic representation of four satellites and the GPS receiver system 


This section represents the family member recognition procedure of our proposed guided glass 
system. We used the faster R-CNN method for the recognition process faster region recognition then applies 
a CNN which makes the recognition process high accuracy. Figure 5 represents the training and real-time 
family member recognition method of our proposed system. 

We use the inception v3 model in training. The model was fine-tuned by the dataset. In faster 
R-CNN [24], [25], the very first thing was region proposal network (RPN). In the beginning, it found out the 
face within the image then apply CNN classification to it. So, for the train of the network, we used 2,280 
images with 5 different categories. Among the images 452 images were the happy face, 402 images were the 
sad face, 432 images were the worried face, 502 images were the surprised face and 492 images were the 
neutral face. The ratio of training, testing, and validation were 4:1:1. Figure 6 represents our used training 
images with xml file. The xml file, containing the level mapping information as name, height, the width of a 
picture. Figure 7 represents the total loss of the training process. After about 20,000 iterations the loss was 
less than 0.02. So, our training was good as its normal standard is 0.05. 
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Figure 5. Flow chart of real-time recognition of family members 
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Figure 6. Represents training images with the xml file 
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Figure 7. Total loss during the training, where the X-axis represents the number of iterations and the Y-axis 
represents the amount of loss 
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3. RESULTS AND DISCUSSION 

Our smart glass successfully detected the barrier and notified the blind man as shown in Figures 8. 
Here, Figure 8(a) represents plane surface and Figure 8(b) barrier in front of the man. It was found that 2 cm 
to any deep and barricades ahead of the visually handicapped person. We showed that our guiding glass 
found objects before the blind person within a distance of 2 to 400 cm. When our system found a car about 
200 cm in front of it, then it automatically recognized the car and sent audio instruction “an object in front of 
you about 2 meters distance, and it is a car” to the ears of the visually impaired people. Object identification 
are shown in Figure 9, where Figures 9(a) to 9(c) represents practically object recognition at different places 
by using our module. 


(a) (b) 


Figure 8. Represents barrier and hole detection performance analysis of our module (a) on the plane surface 
(not any audio instructions) and (b) barrier in front of the man (“a barrier in front of you” instruction) 


(a) (b) (c) 


Figure 9. Real-time object detection by using proposed glass module at different places (a) “a car in front of 
you” audio instructions, (b) “a chair in front of you” audio instructions, and (c) “a dustbin in front of you” 
audio instruction 


Table 2 represents object identification in various places on the university campus. Here we give 
some real-time recognition data with associated distance information. Mail information was “I am in trouble; 
please help me. My location is 22.09829T89.50255”. The mail represented the longitude and latitude value, 
and family members found out the blind individual within a second by using Google Maps. The mail can be 
shown as in Figures 10. Here, Figure 10(a) represents snapshot of email information, Figures 10(b) and 10(c) 
also represents different location information of the blind individual which was sent to the family member. 
Using Google Map, the family member could also know the distance in meters and way distance in time by 
walk, train, bus. 

The GPS data was taken using the proposed guiding glass in various locations. The information was 
taken from the new academic building blocks (Newacd-blocks). GPS information in the new academic 
building of our guiding glass and google actual values are given in Table 3 and displayed in Figures 11, 
where Figure 11(a) represents actual location value plot and Figure 11(b) actual and module location 


Int J Elec & Comp Eng, Vol. 12, No. 5, October 2022: 5543-5552 


Int J Elec & Comp Eng ISSN: 2088-8708 O 5549 


information of vision impairment individual. GPS information from our proposed guiding glass has a little 
different than the actual location found on the Google map. Our A, B, C, D blocks locations have about 3 m, 
55 m, 79 m, and 38 m values different from the actual location. We found a percentage error of about 
0.0009% to 0.004% during A, B, C, D blocks. 

Our system successfully recognized the person in front of the glass. Using faster R-CNN its 
recognition accuracy was very high compared to SVM, R-CNN, KNN CNN, and Fast R-CNN. For family 
member recognition, the system accuracy was 98.89%. Figure 12 represents family member recognition with 
name information in Figures 12(a) and 12(b) known faces with name information. Figure 12(c) an unknown 
marked face that was not present in the module database. These recognition results were produced audio 
information like “Thoha’ in front of you”. In this way, the blind individual knew who was in front of him or her. 


Table 2. Represents different object detection associated with audio instructions 


SL# Original — identified object Obstacle distance (cm) Voice instruction 
01 Laptop —Laptop 300 The object in front of you is about “3” meters and it is a “laptop” 
02 Table —Chair 208 --- “2 “meters and it is “chair” 
03 Mobile Cell phone 300 --- “3 “meters and it is “cell phone” 
04 Desktop Monitor 200 --- “2 “meters and it is “monitor” 
05 Fan — Electric fan 300 --- “3 “meters and it is “electric fan” 
06 Book —Book 360 --- “3 and a half meter” and it is “book” 
07 Bottle— Water bottle 400 --- “4 “meters and it is “water bottle” 
08 mouse —Mouse 230 --- “2 “meters and it is “mouse” 
09 Pen —Fountain pen 100 --- “1“meters and it is “fountain pen” 
10 Lighter Lighter 400 --- “4“meters it is “lighter” 
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Figure 10. Performance analysis of GPS features of our proposed blind glass module (a) snapshot of email 


information, (b) and (c) snapshots of location information of the blind man which was sent to the family 
member 


Table 3. Module accuracy identification in the academic building (Newacd-blocks) 


Location Module GPS values Actual GPS values from Google map Percentage error 
Latitude Longitude Latitude Longitude 
Newacd -A 22.89923 89.50200 22.89922 89.50200 0.0009% 
Newacd-B 22.89899 89.50166 22.89845 89.5016 0.002% 
Newacd -C 22.89917 89.50146 22.89834 89.5015 0.004% 
Newacd -D 22.89893 89.50120 22.89854 89.5015 0.002% 
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(b) 
Figure 11. Represents snapshots of A, B, C, D block GPS location (a) actual location and (b) actual location 
and module GPS location plot information of vision impairment individual 


oo 
(b) 
Figure 12. Represents real-time face recognition by using a PI camera after completing the training during 


software build-up section (a) known face “Thoha”, (b) known face Toufik, and (c) face was not present in the 
database so recognized by unknown face 
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4. CONCLUSION 

We introduced a smart guiding glass that can provide necessary information to visually impaired 
people. Using a machine-learning algorithm, the smart guiding glass can quickly identify all types of objects, 
and the person who was in front of them. The audio instructions were effectively sent to the ears of the 
visually impaired individual. We hope that our proposed guiding glass would be much helpful for blind 
individuals. In addition, the location tracking would also be beneficial for the blind individual indoors and 
outdoors atmosphere. We believe that the government or high-tech company will build this product 
commercially for blind people. 
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